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VISUALIZATION  SUPPORT  FOR  AN  ARMY  RECONNAISSANCE  MISSION 


Tom  Witte  and  Valerie  C.  Kelly 
U.S.  Army  Topographic  Engineering  Center 
Alexandria,  Virginia  22310-3864 

ABSTRACT 

The  unmanned  aerial  vehicle  fUAV)  is  gaining  Importance  as  a 
means  of  acquiring  real-time  information  under  difficult 
environmental  and  battlefield  conditions.  In  Operation 
Desert  Storm,  UAVs  were  effective  in  gathering  information 
and  in  searching  for  mobile,  semi-permanent  assets.  UAV 
systems  provided  reconnaissance .  surveillance,  target 
acquisition,  and  battle  damage  assessment.  A  recent  Army 
project  Integrated  GPS  tracking,  video  imaging,  and  3-D 
visual  simulation  in  an  investigation  of  their  benefits  to 
the  usefulness  of  UAVs.  Construction  of  the  simulation 
database  entailed  the  use  of  digital  stereophotogrammetric 
techniques  to  orthorectlfy  USGS  NAPP  photographs.  Numerous 
digital  orthophotographs  were  mosaicked  together  and  used  as 
texture  atop  digital  terrain  elevation  data  to  generate  the 
simulated  environment.  An  experiment  was  conducted  in  which 
a  UAV  flew  over  a  test  site,  providing  video  imagery  as  well 
as  reporting  position  and  orientation  to  ground  observers. 
The  video  Imagery  was  compared  with  a  computer  simulation  of 
the  Identical  flight  path.  Other  issues  to  be  addressed 
included  the  ability  to  control  the  UAV  via  uploading  of 
waypoints  chosen  from  the  computer  simulation,  the  ability 
to  track  a  ground-based  target  vehicle,  and  the  ability  to 
plan  UAV  missions  rapidly  via  simulation. 

INTRODUCTICm 

The  cost,  complexity  and  environmental  impact  of  live 
training  exercises  make  simulation  increasingly  attractive. 
Pilots  and  tank  drivers  can  use  computer-generated  terrain 
scenes  during  training  to  save  resources  and  provide 
experience  in  potentially  dangerous  or  difficult  situations. 
These  scenes  are  produced  using  advanced  computer  graphics 
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technology  and  digital  topographic  data.  The  U.S.  Army 
Topographic  Engineering  Center  (TEC)  Is  researching  ways  to 
Improve  this  technology  and  apply  It  to  military  mission 
planning  and  rehearsal,  battlefield  management  and  advanced 
weapons  design. 

TEC'S  Battlefield  Visualization  Test  Bed  (BVTB)  is  an 
assembly  of  hardware  and  software  used  to  develop  and 
demonstrate  visualization  technology.  The  BVTB  Is  designed 
to  support  the  scientific  Investigation  of  visualization 
applications  and  techn^^logy  transfer  Issues. 

An  Ideal  visualization  system  provides  a  realistic  view  of 
the  battlefield,  allowing  soldiers  to  see  hills,  trees, 
roads,  waterways,  etc.  The  technology  also  enables  the 
commander  to  create  and  represent  complex  battlefield 
situations  rapidly  and  effectively.  With  battlefield 
visualization  capabilities,  the  commander  can  thoroughly 
examine  the  terrain  and  quickly  evaluate  possible  battle 
scenarios  using  simulated  plans  and  maneuvers. 

Today's  command,  control  and  communications  systems  require 
an  Integrated  simulation  capability  to  support  the  commander 
In  his  real-time  decision  making.  Researchers  at  TEC  will 
continue  to  use  the  BVTB  to  develop  new  scene  generation 
techniques,  and  to  transfer  and  Integrate  battlefield 
visualization  capabilities  into  Army  programs.  This  will 
help  the  Army  train  more  effectively,  develop  weapons 
systems  smarter  and  faster,  plan  and  rehearse  tactical 
operations  more  quickly,  and  enable  the  commander  to  see  and 
understand  complex  battlefield  situations. 

BACKGROUND 


Among  the  most  significant  lessons  of  Operation  Desert  Storm 
was  recognition  of  the  need  for  real-time  battlefield 
intelligence.  UAVs  are  emerging  as  cost  effective  systems 
for  providing  vital  Intelligence  In  real-time  and  at  no  risk 
to  soldiers  and  equipment.  In  Operation  Desert  Storm,  UAVs 
were  especially  effective  In  gathering  Information  and  In 
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searching  for  mobile,  semi -permanent  assets.  These  systems 
provided  reconnaissance,  surveillance,  target  acquisition 
and  battle  damage  assessment  (Defence  1993). 

The  Joint  Precision  Strike  program  seeks  to  reduce  the  time 
required  for  reconnaissance,  surveillance,  strike  planning, 
weapons  delivery  and  battle  damage  assessment.  TEC  recently 
participated  in  a  Joint  Precision  Strike  demonstration 
Involving  UAVs.  Concepts  to  be  tested  Included  satellite 
control  of  a  UAV  and  Its  on-board  EO  sensor,  cross 
correlation  of  live  versus  synthetic  scenes  and  real-time 
synthetic  representations  of  target  acquisition  and  weapons 
delivery. 

In  this  experiment,  a  GPS  receiver  and  an  EO  sensor  were 
mounted  on  a  UAV  which  flew  over  a  target  set,  providing 
telemetry  every  second.  The  UAVs  altitude  and  speed  were 
to  be  held  constant.  This  Information  was  to  be  transmitted 
to  TEC  and  used  to  animate  the  viewpoint  of  a  visual 
simulation  software  program  that  was  simultaneously 
navigating  a  high  resolution,  3-D  database  of  the  same  area. 
The  database  consisted  of  elevation  data  textured  with 
orthorecti fled  aerial  photographs.  The  UAV  also  was  to 
transmit  Imagery  with  annotated  telemetry  data  to  a  ground 
station.  Live  vldeolmagery  was  to  be  compared  with  the 
simulated  scenes. 


DATABASE  CONSTRUCTION 


Source  acquisition 

The  location  for  the  test  was  a  lO-mlnute-by-lO-mlnute  area 
In  the  Mojave  Desert,  southeast  of  Edwards  Air  Force  Base  In 
southern  California.  The  site  falls  within  the  Adobe 
Mountain,  Shadow  Mountains,  Shadow  Mountains  SE  and  El 
Mirage  quadrangles  produced  by  the  U.S.  Geological  Survey 
(USGS).  These  maps  were  based  on  the  North  American  Datum 
of  1927  and  the  National  Geodetic  Vertical  Datum  of  1929. 

To  provide  full  areal  photographic  coverage  of  the  site  at 
high  resolution,  21  9x9  National  Aerial  Photography  Program 
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(HAPP)  photographs  were  purchased  from  the  USGS.  Three 
flight  strips  of  seven  photographs  each  covered  the  test 
site.  As  digital  coverage  was  not  available,  film  positives 
sufficed.  NAPP  standards  specify  quarter  quadrangle- 
centered  photographs,  exposed  at  a  height  of  20,000  feet 
above  the  ground,  using  a  six  inch  focal  length.  These 
1:40,000  scale  photographs  were  scanned  at  60  microns  to 
produce  digital  images  with  a  ground  resolution  of  2.4 
meters  per  pixel. 

On  the  NAPP  imagery,  of  course,  points  on  the  ground  are 
displaced  as  a  function  of  their  height.  These  relief 
displacements  prohibited  the  determination  of  accurate 
linear  measurements,  which  were  required  for  the  experiment. 
Accurate  measurements  can  be  made  only  on  orthoimages  of  the 
original  photographs.  The  goal  of  digital  orthophotography 
is  to  present  each  pixel  as  if  it  were  viewed  from  directly 
above,  with  the  camera  orthogonal  to  the  ground. 

Orthorectification 

A  digital  elevation  model  (DEM)  is  needed  to  transform  an 
aerial  photographic  image  into  an  orthophoto  image.  In  this 
process,  the  horizontal  coordinates  and  elevation  of  a  point 
on  the  ground  are  used  to  determine  the  position  of  the 
point  on  the  original  photograph.  The  gray  shade  assigned 
to  that  position  in  the  digital  image  is  then  assigned  to 
the  corresponding  horizontal  position  on  the  orthophoto. 

The  result  is  a  new  digital  image  without  relief 
displacement.  This  orthophoto  image  will  benefit  from  the 
appearance  of  a  photograph  and  the  geometric  properties  of  a 
topographic  map.  These  characteristics  enable  the  image 
data  to  be  registered  to  the  elevation  data;  this  is 
essential  for  an  accurate,  realistic  fly-through. 

Horizontal  control  points  were  selected  from  the  four  USGS 
1:24,000  scale  topographic  maps  covering  the  test  range. 
These  maps  have  a  stated  absolute  accuracy  of  12  meters  at 
90  percent  circular  error.  Defense  Mapping  Agency  (DMA) 
Digital  Terrain  Elevation  Data  (DTED)  provided  the  vertical 
control.  Level  1  DTED  (a  unifom  matrix  of  elevation  values 
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spaced  3  arc  seconds  apart)  was  used.  The  absolute  vertical 
accuracy  objective  for  DTED  Level  1  is  +  30  meters  at  90 
percent  linear  error. 

The  digital  orthorectification  was  performed  by  the  Imagery 
Extraction  Division  (lED)  using  TEC's  Terrain  Information 
Extraction  System  (TIES) .  TIES  is  an  integrated  digital 
stereo  image-based  mapping  system  that  uses  photograimietrlc 
CIS  technology.  One  of  the  key  components  of  TIES  is  the 
Digital  Stereo  Photogrammetric  Workstation  (DSPW),  a 
developmental  system  which  can  be  used  to  manage  and  display 
digital  Imagery  and  its  geometric  support  data, 
aerotriangulate  digital  Imagery,  rectify  digital  imagery, 
automatically  create  digital  terrain  models  (DTM)  at  any 
specified  elevation  point  spacing,  interactively  edit  DTMs 
in  stereo,  and  to  display  and  interactively  edit  digital 
feature  data  that  are  stereoscopical ly  superimposed  on 
stereo  imagery. 

Another  component  of  TIES,  the  image  Digitizing  System  (IDS) 
was  used  to  convert  the  MAPP  photographs  into  digital 
Images.  The  IDS  scanner  is  a  precise  continuous  scanning 
optical  device  consisting  of  a  2,048-element  array  of 
charge-coupled  micro-sensors.  The  scanning  resolution  is 
chosen  to  equal  the  product  of  7.5  microns  and  powers  of  two 
(e.g.  7.5,  15.0,  30.0,  60.0,  120.0,  etc.).  For  this  project, 
the  Imagery  was  scanned  at  a  resolution  of  60  microns  and 
then  written  to  an  8mm  tape  for  loading  onto  the  DSPW. 

Camera  calibration  data  were  also  recorded  at  this  time. 

The  scanning  process  converts  raw  photos  into  pixels,  each 
of  which  has  an  address  location  in  the  image  in  terms  of 
rows  and  columns.  However,  this  location  is  a  distortion  of 
the  true  ground  location  of  the  area  represented  by  the 
pixel . 

The  digital  Imagery  was  Imported  into  the  DSPW  via  a  frame 
liqport  function.  The  image  files  and  their  associated 
support  data  were  loaded,  control  points  were  measured  and 
exterior  orientation  was  performed.  The  points  were 
measured  using  ERDAS  software  and  a  digitizing  table. 
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Corresponding  image  points  were  measured  and  stored  In  a 
file. 

A  least  squares  adjustment  was  performed  to  determine  the 
optimum  sensor  model  to  fit  the  measured  points.  Due  to 
time  constraints,  a  bundle  adjustment  was  performed  on  each 
of  the  three  strips  rather  than  on  the  entire  21-photo 
block.  This  established  the  orientation  of  the  ceusera 
relative  to  the  ground.  Upon  successful  completion  of  the 
triangulation  process,  the  orthophotos  were  generated.  The 
generation  of  one  orthophoto  for  every  other  photo  was 
sufficient  to  ensure  complete  coverage  of  the  test  region. 

Thus,  four  orthophotographs  were  generated  per  flight  strip. 
These  four  digital  orthophotographs  were  added  together  to 
form  four  rows  running  east-west.  These  rows  were  digitally 
mosaicked  together  from  north  to  south,  as  follows:  The 
first  and  second  rows  were  mosaicked  together  to  form  one 
top  row,  and  the  third  and  fourth  rows  were  likewise  joined 
to  form  a  single  bottom  row.  These  top  and  bottom  rows  were 
finally  mosaicked  together  to  form  a  complete, 
orthorecti fled  Image  of  the  test  site.  The  mosaicking 
process  Involved  some  trimming  away  of  overlapping  Imagery. 
Image  enhancement  was  not  deemed  necessary  for  the  purposes 
at  hand,  and  was  not  performed. 

The  Image  data  were  given  to  the  operator  of  the 
visualization  software  on  a  single  8oim  tape  which  contained 
twelve  Individual  digital  orthophotographs  at  full 
resolution,  and  one  file  containing  a  one-fourth  down- 
sampled  Image  of  the  entire  test  site.  The  Individual 
orthophotographs  were  slightly  less  than  one  megabyte  each, 
and  the  overview  took  up  about  four  megabytes.  The  operator 
needed  only  to  know  the  pixel  size,  the  number  of  rows  and 
columns,  and  the  coordinates  of  the  southwest  corner  of  each 
Image  in  order  to  prepare  for  the  fly-through  process. 

SCENE  SIMULATION 


Texture 
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Texture  mapping  is  a  technique  used  to  add  realism  and 
detail  to  Images.  A  texture  pattern  Is  an  Image  that  can  be 
ai^lled  to  a  polygon  as  wallpaper  Is  applied  to  a  wall.  In 
3-D  graphics,  these  patterns  provide  the  correct 
perspective,  appearing  to  stay  glued  to  the  underlying 
surfaces  as  the  viewpoint  changes.  More  specifically,  a 
texture  Is  a  2-D  array  of  pixel  Information.  Texture 
coordinates  can  be  assigned  to  the  vertices  of  polygons. 

The  system  can  then  Interpolate  texture  coordinates  as  It 
fills  the  polygons. 

The  project  described  herein  involved  the  use  of  geospeclflc 
texture  -  aerial  Imagery  of  the  test  range  -  as  a  pattern  to 
cover  the  polygons  defined  by  the  terrain  elevation  data  In 
a  simulation.  The  Imagery  was  divided  Into  many  Individual 
tiles  which  were  then  applied  to  the  polygons  that  were 
approximating  the  DTED-based  terrain.  The  result  was  a 
photo-reallstlc  database  that  provided  good  visual  cues  for 
depth  and  height.  However,  texture  mapping  Is  very 
demanding  of  a  system's  speed  and  memory  (Latham  1992). 

The  software  used  for  mission  planning,  observation  and 
play-back  was  DrawLand,  an  In-house  program  used  to  support 
terrain  visualization  research.  DrawLand  behaves  like  a 
flight  simulator;  what  Is  seen  on  the  monitor  changes  in 
response  to  how  one  manipulates  (via  mouse  or  other  Input 
device)  the  controls  of  a  virtual  vehicle.  However,  the 
program  does  not  attempt  to  mimic  the  flight  dynamics  of  a 
specific  aircraft. 

DrawLand 's  main  Input  Is  DTED,  which  the  program  can  load 
Into  memory  directly  from  CD-ROM.  The  user  can  specify  the 
area  to  be  loaded.  Given  the  southwest  corner  and  the  size 
of  the  cell,  DrawLand  searches  for  the  appropriate  files, 
and  can  combine  data  from  several  files  If  necessary. 

DrawLand 's  Interactive  or  flight  mode  begins  by  displaying 
the  terrain  as  a  one-third  resolution  wireframe.  The  user 
can  adjust  the  resolution  at  will  and  also  vary  the 
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rendering  method  to  suit  the  purpose  (e.g.,  wireframe  for 
speed,  photo-texture  for  realism) .  The  viewpoint  can  be 
controlled  by  a  mouse.  Coordinates,  altitude  above  MSL, 
distance  above  ground,  yaw,  pitch  and  roll  are  displayed  for 
the  viewpoint  or  for  a  model. 

During  flight  mode,  the  movement  of  the  user's  vle%#polnt  can 
be  recorded,  as  can  the  movement  of  a  particular  model  that 
Is  being  controlled.  The  animation  can  be  recorded  one 
frame  at  a  time  to  an  optical  disk  or  to  a  hard  disk.  Model 
movements  can  also  be  played  back  in  real  time. 

EXPERIMENT 

The  UAV  flew  on  the  western  part  of  the  range  In  a  holding 
pattern.  There  were  two  potential  targets  -  a  tank  and  a 
mobile  surface-to-surface  missile  launcher.  Special  forces 
on  the  ground  were  to  locate  targets  and  then  transmit  their 
coordinates  to  the  mission  planner  operating  DrawLand 
visualization  software  remotely  at  USATEC.  Icons 
representing  the  targets  were  to  be  Inserted  Into  the 
synthetic  scene  In  their  proper  positions  to  facilitate 
mission  planning,  targeting  and  weapons  delivery  In  the 
synthetic  environment.  Upon  Identification  of  a  target,  the 
mission  planner  would  define  and  transmit  accurate  waypoints 
for  a  valid  flight  path  toward  the  target  back  to  the  ground 
station  In  California  for  subsequent  uplink  to  the  UAV. 

It  was  planned  that  the  telemetry  would  be  sent  from  the  UAV 
to  a  coimnunlcatlons  satellite,  which  would  transmit  It  to 
the  ground  station,  which  would  then  send  It  to  TEC  In  ASCII 
format  via  a  T1  line.  The  coordinates  of  waypoints  were  to 
be  sent  via  T1  line  from  TEC  to  the  ground  station,  then 
passed  to  the  satellite  and  back  to  the  UAV.  Videoimagery 
was  to  be  transmitted  from  the  UAV  to  the  satellite  and  then 
split  to  both  the  ground  station  and  TEC.  Concepts  to  be 
tested  Included  non-llne-of-slght  control  for  the  UAV  as 
well  as  the  Incorporation  of  actual  targets  Into  a  synthetic 
environment  for  command  and  control  purposes.  Video  and 
synthetic  Imagery  were  to  be  compared,  and  the  usefulness 
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and  validity  of  live  video  Imagery  were  to  be  assessed. 

RESULTS 

The  UAV  flew  on  more  than  one  occasion;  no  single  flight 
addressed  all  of  the  salient  Issues.  Real-time  tracking  and 
video  transmission  did  not  occur  simultaneously.  There  were 
some  problems  with  the  data  received  from  the  UAV,  and 
trouble  was  also  encountered  In  configuring  the  ports  on  the 
visualization  workstation.  Vldeolmagery  and  telemetry  were 
transmitted  from  the  UAV  to  the  satellite  and  to  TEC  In  real 
time;  however,  visualization  techniques  were  not  employed  on 
that  particular  date.  On  those  occasions  when 
vlsuallzatlon-alded  mission  planning  was  to  be  conducted 
rMiotely  from  TEC,  both  the  target  coordinates  and  waypoint 
coordinates  were  to  be  passed  via  cellular  STU-III,  a  secure 
telephone  with  a  built-in  modem.  However,  although  the 
capability  existed.  It  was  not  used. 

GPS  receiver  log  data  were  sent  by  modem  to  a  PC  and  were 
do«mloaded  for  subsequent  use  by  DrawLand.  The  data  packets 
transmitted  by  the  ground  control  station  contained  the 
UAV's  latitude,  longitude  and  altitude,  as  well  as  time  of 
fix,  pitch,  roll,  heading,  airspeed,  etc.  Three  sets  of 
data  were  sent  to  support  visualization  efforts:  the  first 
set  was  completely  bad,  and  the  other  two  contained  correct 
latitudes  and  longitudes  only. 

Black  and  white  Imagery  from  an  early  trial  run  was  recorded 
on  videotape  and  brought  to  TEC  for  comparison  with 
simulated  flights  over  the  same  area.  The  quality  of  this 
vldeolmagery  was  generally  poor,  and  some  new  structures 
appeared  which  were  not  present  In  the  synthetic  scene,  due 
to  the  date  of  acquisition  of  the  HAPP  Imagery.  However, 
the  DrawLand  operator  was  able  to  duplicate  the  UAV's  flight 
path  for  a  slde-by-slde  comparison.  The  simulation  actually 
was  visually  superior,  due  In  part  to  the  poor  quality  of 
the  video,  and  to  the  fact  that  the  haze  and  fog  that 
existed  at  the  test  range  were  not  rendered  by  the  computer. 
A  higher  quality  color  video  was  recorded  later  In  the 
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testing  process,  but  It  wasn't  used  to  recreate  the  UAV's 
flight. 


CONCLUSION 

The  concept  of  non-llne-of-slght  control  for  a  UAV  and  Its 
on-board  sensor  was  successfully  demonstrated,  as  was  the 
use  of  advanced  visualization  techniques  to  Improve  the  UAV 
mission  planning  process.  The  application  of  real-time 
tracking  and  control  using  visualization  via  direct 
satellite  link  will  extend  the  ranges  of  UAVs  and  Improve 
their  effectiveness  to  the  Army.  Further  testing  and 
experimentation  will  be  done  In  this  area.  Researchers  at 
USATEC  will  continue  to  develop  simulation  capabilities  and 
to  Integrate  them  effectively  Into  Army  systems  and 
programs.  UAV  systems  will  continue  to  be  used  to  acquire 
real-time  reconnaissance.  Intelligence  and  targeting 
Information,  not  only  for  locating  high-value  military 
targets,  such  as  Scud  missiles,  but  also  for  applications 
such  as  treaty  verification  and  compliance.  There  are 
numerous  potential  commercial  applications,  as  well. 
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